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Program description 


Too Good for Drugs™ is designed to promote life skills, char- 
acter values, resistance skills to negative peer influence, and 
resistance to the use of illegal drugs, alcohol, and tobacco. The 
program, which targets elementary and middle school students, 
is based on classroom discussions and structured activities 
that center on interactive learning and skill-building exercises. 


Students engage in role-play and cooperative learning games 
and are encouraged to apply the skills to different contexts. 

Too Good for Drugs™ also includes the optional elements of 
parental and community involvement. Two related programs are 
addressed in the intervention reports on Too Good for Druas and 
Violence fhiah school) and Too Good for Violence (K-81. 


Research 


Two studies of Too Good for Drugs™ met the What Works 
Clearinghouse (WWC) evidence standards. These studies, which 
included nearly 2,500 students in grades 3, 4, and 6, attending 


12 elementary and middle schools in Florida, examined results 
on students’ behavior and knowledge, attitudes, and values. 1 



Effectiveness Too Good for Drugs™ was found to have potentially positive effects on students’ behavior and no discernible effects on students’ 
knowledge, attitudes, and values. 





Behavior 


Knowledge, attitudes, and values 


Academic achievement 


Rating of effectiveness 


Potentially positive effects 


No discernible effects 


Not reported 


Improvement index 2 


Average: +10 percentile points 
Range: 0 to +17 percentile 
points 


Average: +7 percentile points 
Range: +3 to +11 percentile points 


Not reported 



1. The evidence presented in this report is based on the available research. Findings and conclusions may change as new research becomes available. 

2. These numbers show the average and range of improvement indices for all findings across the two studies. 
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Developer and contact 

Mendez Foundation. 601 S. Magnolia Avenue, Tampa, FL 33606. 
Web: www.mendezfoundation.orc . Telephone: 800-750-0986. 

Scope of use 

Too Good for Drugs™ (K-8) was first developed in Flillsborough 
County (Tampa), Florida in 1978. The middle school program 
was revised in 1995. 3 Too Good for Drugs™ and its companion 
programs (Too Good for Violence and Too Good for Drugs and 
Violence) have been implemented in more than 2,500 districts in 
more than 48 states in rural, urban, and suburban communities 
with African-American, Asian, Hispanic, and Caucasian student 
populations and across diverse socioeconomic groups. Too 
Good for Drugs™ may have changed since the studies were 
conducted. The WWC recommends asking the developer for 
information about the most current version of this curriculum 
and taking into account that student demographics and school 
context may affect outcomes. 

Teaching 

Too Good for Drugs™ was included in the character education 
review because the program addresses several character traits 
that are infused into most of the lessons. Too Good for Drugs™ 
consists of 10 lessons at each grade level lasting 30-45 minutes 
per lesson. All lessons are scripted and intended to be taught by 
trained teachers or program instructors (off-site educators). Les- 
sons include information about the frequency of drug use among 
American youth and the harmful effects of drug use. Instructional 
strategies cover goal setting and decisionmaking skills, prosocial 
skills, resistance to negative peer influence skills, and interpersonal 



skills. Core values such as respect for self and others, empathic 
responding, and responsibility are integrated into the lessons. 
Cooperative learning activities, role-play, and skill-building meth- 
ods reinforce positive behaviors and skills and encourage students 
to apply these behaviors and skills in other contexts. 

The developer provides such teacher resources as grade- 
level kits that include scripted curricula, 50 student workbooks, 
measurable objectives, evaluation tools, lesson extenders, and 
tips for teaching the program. According to the developer, the 
program is school-based but also includes such optional com- 
munity and parental involvement components as parent newslet- 
ters and interactive family materials as well as information on 
holding parent information sessions. 

Cost 

The cost of materials for a classroom, including the curriculum, 
50 student workbooks, teaching materials such as puppets and 
posters, teaching tips, and evaluation tools, ranges from $100 to 
$130 depending on the grade level. 

Teachers are encouraged to attend an on-site or regional 
curriculum training workshop held by the developer. The cost 
per day of a regional training workshop is $300 a person for 
curriculum training and $400 a person for train the trainer ses- 
sions. The cost of the regional training is reduced to $850 if the 
participant attends all three days of training. The cost per day 
of an on-site training workshop, which can train groups of 15 to 
50 participants, is $1,500 plus travel for curriculum training and 
$225 a person for train the trainer sessions. The developer states 
that smaller school districts may collaborate with nearby districts 
to share the cost of on-site training. 



Two studies reviewed by the WWC investigated the effects of 
Too Good for Drugs™. Both studies (Bacon, 2000; Bacon, 2003) 
were randomized controlled trials that met WWC evidence stan- 
dards. Both studies focused on Too Good for Drugs™ imple- 
mented in classrooms rather than as a schoolwide intervention. 

3. The revised middle school curriculum is also known as Too Good for Drugs II. 



The Bacon (2000) study investigated the program effects 
on middle school students and included more than 1,300 
sixth-grade students attending six middle schools in one large 
school district in Florida. This study compared outcomes for 
students participating in a Too Good for Drugs™ curriculum with 



2 




Research (continued) 



Effectiveness 



The WWC found Too Good for 
Drugs™ to have potentially 
positive effects on behavior 
and no discernible effects 
on knowledge, attitudes, 
and values 
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outcomes for students in classes that did not use a character 
education curriculum. 

The Bacon (2003) study investigated the program effects on 
elementary school students and included more than 1,100 third- 
and fourth-grade students attending six elementary schools in 



one school district in Florida. This study compared outcomes for 
students participating in a Too Good for Drugs™ curriculum with 
outcomes for students in classes that did not use a character 
education curriculum. 



Findings 

The WWC review of character education addresses student 
outcomes in three domains: behavior; knowledge, attitudes, and 
values; and academic achievement. 

Behavior. Bacon (2003) found statistically significant differ- 
ences favoring the intervention group on all three subscales 
(personal and social skills, prosocial behavior, and inappropriate 
behavior) four months after the end of the program. 4 Although, 
as calculated by the WWC, none of these outcomes — individu- 
ally or averaged — were found to be statistically significant the 
average effect size was large enough to be considered substan- 
tively important (at least 0.25). 

Knowledge, attitudes, and values. Bacon (2000) reported 
statistically significant differences favoring the intervention group 
on three outcomes (resistance skills, prosocial peer group, and 
locus of control) four months after the end of the program. But 
none of these outcomes were found to be statistically significant 
as calculated by the WWC. The average effect size was neither 
statistically significant nor substantively important. 



Bacon (2003) reported statistically significant differences in 
student perceptions favoring the intervention group on one of the 
five outcomes (goal setting and decisionmaking) four months after 
the end of the program. This effect was not found to be statistically 
significant as calculated by the WWC. The average effect size was 
neither statistically significant nor substantively important. 

Rating of effectiveness 

The WWC rates interventions as positive, potentially posi- 
tive, mixed, no discernible effects, potentially negative, or 
negative. The rating of effectiveness takes into account four 
factors: the quality of the research design, the statistical 
significance of the findings (as calculated by the WWC), the 
size of the differences between participants in the intervention 
condition and the comparison condition, and the consistency 
of the findings across studies (see the WWC Intervention Rat- 
ing Scheme) . 



Improvement index 

For each outcome domain, the WWC computed an improvement 
index based on the average effect size (see the WWC Improve- 
ment Index Technical Paper) . The improvement index represents 
the difference between the percentile rank of the average 
student in the intervention condition versus the percentile rank of 
the average student in the comparison condition. Unlike the rat- 
ing of effectiveness, the improvement index is entirely based on 



the size of the effect, regardless of the statistical significance of 
the effect, the study design, or analysis. The improvement index 
can take on values between -50 and +50, with positive numbers 
denoting favorable results. The average improvement index for 
behavior is +10 percentile points, with a range of 0 to +17 per- 
centile points across findings. The average improvement index 
for knowledge, attitudes, and values is +7 percentile points, with 
a range of +3 to +11 percentile points across findings. 



4. The level of statistical significance was calculated by the WWC and, where necessary, corrects for clustering within classrooms or schools, and for 
multiple comparisons. For an explanation, see the WWC Tutorial on Mismatch . See the WWC Intervention Rating Scheme for the formulas the WWC used to 
calculate the statistical significance. In the case of the Too Good for Drugs™ report, corrections for clustering and multiple comparisons were needed. 
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The WWC found Too Good for 
Drugs™ to have potentially 
positive effects on behavior 
and no discernible effects 
on knowledge, attitudes, 
and values (continued) 



Summary 

The WWC reviewed two studies on Too Good for Drugs™. Both 
studies (Bacon, 2000; Bacon, 2003) were randomized controlled 
trials that met WWC evidence standards. Only one study (Bacon, 
2003) examined student outcomes in the behavior domain. The 
average effect size across all behavior outcomes examined in this 
study was substantively important but not statistically significant. 
So the WWC rated the program as having potentially positive 
effects in the behavior domain. Both studies reviewed for Too 
Good for Drugs™ examined student outcomes in the knowledge, 



attitudes, and values domain. When the WWC aggregated the 
results across all outcomes in this domain in each of the studies, 
the domain average effect size in each of the studies was neither 
statistically significant nor substantively important. In addition, 
none of the individual findings was statistically significant, as 
calculated by the WWC. So the WWC rated the program as hav- 
ing no discernible effects on knowledge, attitudes, and values. 
Character education, an evolving field, is beginning to establish 
a research base. The evidence presented in this report is limited 
and may change as new research emerges. 



References 



Met WWC evidence standards 

Bacon, T. P. (2000). The effects of the Too Good for Drugs 
prevention program on students’ substance use intentions 
and risk and protective factors. Florida Educational Research 
Council, Inc., Research Bulletin, 31(3 & 4), 1-25. 



Bacon, T. P. (2003). Technical report: Evaluation of the Too Good 
for Drugs Elementary School Prevention Program. A report pro- 
duced for Florida Department of Education Department of Safe 
and Drug-Free Schools. Tallahassee, FL. Available from: The 
Mendez Foundation, 601 S. Magnolia Avenue, Tampa, FL 33606. 



For more information about specific studies and WWC calculations, please see the WWC Too Good for Drugs™ 
Technical Appendices . 
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Appendix 



Appendix A1.1 Study characteristics: Bacon, 2000 (randomized controlled trial) 



Characteristic 


Description 


Study citation 


Bacon, T. P. (2000). The effects of the Too Good for Drugs prevention program on students’ substance use intentions and risk and protective factors. Florida Educational 
Research Council, Inc., Research Bulletin, 31 (3 & 4), 1-25. 


Participants 


The comparison group included 1,318 sixth-grade students from six middle schools. About 51% of the student population in these schools in eligible for participation in the 
free or reduced lunch program. Of the sample, 52% were females, 48% were Caucasian, 33% African-American, 13% Hispanic, and 6% Asian. 


Setting 


One large school district in Florida that serves students from urban, suburban, and rural regions. 


Intervention 


The Too Good for Drugs™ sixth-grade curriculum consisted of nine lesson units averaging 45 minutes in length. The program was implemented during the first quarter of the 
school year. 


Comparison 


The comparison group was drawn from matched schools in the same school district. Comparison group students did not participate in the Too Good for Drugs™ program at 
the time of the study but received this program at the fourth quarter of the school year. 


Primary outcomes 
and measurement 


Students responded to survey items assessing students' intentions to use marijuana and their perceptions of peer resistance skills, positive attitudes toward nondrug use, 
perceptions of peer normative substance use, perceptions of peer disapproval of substance use, prosocial peer relationships, and locus of control. (See Appendix A2.2 for a 
more detailed description of outcome measures.) 


Teacher training 


All lessons were delivered by program instructors (trained off-site educators), so no training of teachers was done. 



Appendix A1.2 Study characteristics: Bacon, 2003 (randomized controlled trial) 



Characteristic 


Description 


Study citation 


Bacon, T. P. (2003). Technical report: Evaluation of the Too Good for Drugs Elementary School Prevention Program. A report produced for Florida Department of Education 
Department of Safe and Drug-Free Schools. Tallahassee, FL. Available from: The Mendez Foundation, 601 S. Magnolia Avenue, Tampa, FL 33606. 


Participants 


The study comparison included 1,142 third- and fourth-grade students from six elementary schools. About 45% of the sample was eligible to participate in the free or reduced 
lunch program. Of the sample, 49% were females, 71% Caucasian, 17% African-American, 10% Hispanic, and 2% other race (Asian; American Indian; multicultural). 


Setting 


The school district was in Lake County, Florida. 


Intervention 


The program was implemented during the first half of the school year. Classroom teachers delivered 10 lesson units averaging 45 minutes in length to students in grades 3 
and 4. Students were also encouraged to participate in "Home Workouts” with their family members to reinforce the lessons. 


( continued ') 
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Appendix A1.2 


Study characteristics: Bacon, 2003 (randomized controlled trial) (continued) 


Characteristic 


Description 


Comparison 


The comparison group was drawn from matched schools in the same school district. Comparison group students did not participate in the Too Good for Drugs™ program at 
the time of the study but received it in the fourth quarter of the school year. 


Primary outcomes 
and measurement 


Students responded to two sets of survey items. Three items were used to gauge students' intentions to drink alcohol and use marijuana within the next 12 months. Nineteen 
additional items were used to assess protective factors associated with youth susceptibility to illicit drugs. The 19 items were grouped into such protective factor subscales as 
perceptions of peer resistance skills, prosocial peer relationships, and locus of control. (See Appendices A2.1 and A2.2 for a more detailed description of outcome measures.) 


Teacher training 


No training information was given other than that in small groups or individually teachers received a brief training refresher. 
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Appendix A2.1 Outcome measures in the behavior domain 



Outcome measure 


Description 


The Teacher Checklist of 
Student Behavior (TCSB): 
personal and social skills 


This scale, developed by the study author, consists of 11 items assessing students’ emotional behavior and interpersonal interactions with peers. The checklist was 
completed for each student individually (as cited in Bacon, 2003). 


The Teacher Checklist of 
Student Behavior (TCSB): 
prosocial behaviors 


This scale, developed by the study author, consists of six items assessing students’ helping, respectful, and emphatic behavior with peers. The checklist was completed for 
each student individually (as cited in Bacon, 2003). 


The Teacher Checklist of 
Student Behavior (TCSB): 
inappropriate behaviors 


This scale, developed by the study author, consists of six items assessing students’ aggressive and disruptive behavior. The checklist was completed for each student 
individually (as cited in Bacon, 2003). 



Appendix A2.2 Outcome measures in the knowledge, attitudes, and values domain 



Outcome measure 


Description 


Intentions for drinking 


One survey item on which students indicate if they intend to drink alcohol anytime during the next year (as cited in Bacon, 2000). 


Intention for marijuana 


One survey item on which students indicate if they intend to use marijuana anytime during the next year (as cited in Bacon, 2000). 


Perceptions of social 
and resistance skills 


A measure on which students indicate if they can tell the difference between healthy and unhealthy relationships and if they are able to avoid unhealthy behaviors (as cited 
in Bacon, 2000; Bacon, 2003). 


Prosocial peers 


A scale composed by the study author for the purpose of this study to assess perceptions of prosocial peer behaviors (as cited in Bacon, 2000). 



Locus of control 


A scale composed by the study author for the purpose of this study to assess perceptions of locus of control related to being able to avoid drinking, fighting, and drug use 
(as cited in Bacon, 2000). 



Perceptions of emotional 
competency skills 


A six-item scale developed by the study author on which students indicated if they felt confident in their ability to manage their behavior and emotions and to successfully 
plan for personal goals (as cited in Bacon, 2003). 


Perceptions of goal setting 
and decisionmaking skills 


A seven-item scale developed by the study author on which students indicated if they managed their actions by setting goals and creating plans to reach these goals (as 
cited by Bacon, 2003). 
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Appendix A3.1 



Summary of study findings included in the rating for the behavior domain 1 









Author’s findings from the study 














Mean outcome 
(standard deviation 2 ) 




WWC calculations 




Outcome measure 3 


Study 

sample 


Sample size 4 
(students/ 
schools) 


Too Good for Comparison 

Drugs™ group group 

(column 1) (column 2) 


Mean difference 5 
(column 1- 
column 2) 


Statistical 
significance 7 
Effect size 6 (at a = 0.05) 


Improvement 

index 8 



Bacon, 2003 (randomized controlled trial) 



Personal and social skills 
(follow-up) 


Grades 3-4 


6/1,051 


3.75 

(0.83) 


3.51 

(0.72) 


0.24 


0.31 


ns 


+12 


Prosocial behaviors 


Grades 3-4 


6/1,051 


3.82 


3.46 


0.36 


0.44 


ns 


+17 


(follow-up) 






(0.86) 


(0.78) 










Inappropriate behaviors 


Grades 3-4 


6/1,051 


4.04 


4.04 


0.00 


0.00 


ns 


+0 


(follow-up) 






(1.04) 


(1.18) 










Domain average 9 for behavior (Bacon, 2003) 










0.25 


ns 


+10 



ns = not statistically significant 



1 . This appendix reports end-of-program and follow-up findings considered for the effectiveness rating and the improvement index. Interim and immediate posttest findings from the same studies are not included in these ratings but are 
reported in Appendix A4.1 . 

2. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. 

3. Bacon (2003) examined effects on students' perceptions of the harmful effects of drugs and alcohol and students’ intentions to smoke tobacco. These outcomes were not included, because they were not relevant to the scope of this 
review. For further information about the scope of this review, please see the Character Education Protocol . 

4. The WWC received confirmation from the study author that the analysis of pretest equivalence was based on the analysis sample rather than the intent-to-treat sample. This analysis addresses concerns about sample attrition that other- 
wise might affect this review. 

5. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 

6. For an explanation of the effect size calculation, please see the WWC Technical Working Paper on Effect Size . 

7. Statistical significance is the probability that the difference between groups is the result of chance rather than a real difference between the groups. The level of statistical significance was calculated by the WWC and, where neces- 
sary, corrects for clustering within classrooms or schools, and for multiple comparisons. For an explanation, see the WWC Tutorial on Mismatch . See the WWC Intervention Rating Scheme for the formulas the WWC used to calculate the 
statistical significance. In the case of the Too Good for Drugs™ report, corrections for clustering and multiple comparisons were needed. 

8. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values 
between -50 and +50, with positive numbers denoting favorable results. 

9. This row provides the study average, which is also the domain average in this case. The WWC-computed domain average effect size is a simple average rounded to two decimal places. The domain improvement index is calculated from 
the average effect size. 
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Appendix A3.2 Summary of study findings included in the rating for the knowledge, attitudes, and values domain 1 









Author’s findings from the study 














Mean outcome 
(standard deviation 2 ) 




WWC calculations 




Outcome measure 3 


Study 

sample 


Sample size 4 5 
(students/ 
schools) 


Too Good for Comparison 

Drugs™ group group 

(column 1) (column 2) 


Mean difference 6 
(column 1- 
column 2) 


Statistical 
significance 8 
Effect size 7 (at a = 0.05) 


Improvement 

index 9 









Bacon, 2000 (randomized controlled trial) 










Intentions for drinking (follow-up) 


Grade 6 


6/1,060 


90 out of 495 
students 


62 out of 298 
students 


1.18 


0.18 10 


ns 


+4 


Intention for marijuana 
(follow-up) 


Grade 6 


6/1,060 


62 out of 510 
students 


54 out of 319 
students 


1.47 


0.23 10 


ns 


+9 


Resistance skills (follow-up) 


Grade 6 


6/1,060 


4.36 

(0.75) 


4.15 

(0.82) 


0.21 


0.27 


ns 


+11 


Peers disapprove use (follow-up) 


Grade 6 


6/1,060 


3.73 

(0.98) 


3.47 

(1.07) 


0.26 


0.26 


ns 


+10 


Prosocial peers (follow-up) 


Grade 6 


6/1,060 


4.58 

(0.74) 


4.50 

(0.86) 


0.08 


0.10 


ns 


+4 


Locus of control (follow-up) 


Grade 6 


6/1,060 


4.37 

(0.63) 


4.25 

(0.72) 


0.12 


0.18 


ns 


+7 


Domain average 11 for knowledge, attitudes, and values (Bacon, 2000) 








0.20 


ns 


+8 








Bacon, 2003 (randomized controlled trial) 










Emotional competency skills 
(follow-up) 


Grades 3-4 


6/935 


4.00 

(0.61) 


3.95 

(0.61) 


0.05 


0.08 


ns 


+3 


Social and resistance skills 
(follow-up) 


Grades 3-4 


6/935 


3.59 

(0.63) 


3.54 

(0.64) 


0.05 


0.08 


ns 


+3 


Goal setting and decision 
making skills (follow-up) 


Grades 3-4 


6/935 


4.33 

(0.73) 


4.21 

(0.71) 


0.12 


0.17 


ns 


+7 


Domain average 11 for knowledge, attitudes, and values (Bacon, 2003) 








0.11 


ns 


+4 


Domain average across studies 












0.16 


ns 12 


+7 



ns = not statistically significant 



1 . This appendix reports end-of-program and follow-up findings considered for the effectiveness rating and the improvement index. Interim and immediate posttest findings from the same studies are not included in these ratings, but are 
reported in Appendix A4.2. 

2. The standard deviation across all students in each group shows how dispersed the participants' outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. 



(continued) 
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Appendix A3.2 Summary of study findings included in the rating for the knowledge, attitudes, and values domain 1 (continued) 



3. Bacon (2003) also examined effects on students’ perceptions of the harmful effects of drugs and alcohol and students’ intentions to smoke tobacco. These outcomes were not included, because they were not relevant to the scope of 
this review. For further information about the scope of this review, please see the Character Education Protocol . 

4. The WWC requested and received from the study author sample sizes for the analysis sample of students for all variables in Bacon (2000) because they were not reported in the study paper. 

5. The WWC received confirmation from the study author that the analysis for pretest equivalence is based on the analysis sample rather than the intent-to-treat sample. This analysis addresses concerns about sample attrition that other- 
wise might affect findings. 

6. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. The numbers in the mean difference column for the intentions for drinking and marijuana use repre- 
sent the odds ratio (ratio between the proportions of the intervention group and comparison group) used to calculate effect size. 

7. For an explanation of the effect size calculation, please see the WWC Technical Working Paper on Effect Size . 

8. Statistical significance is the probability that the difference between groups is the result of chance rather than a real difference between the groups. The level of statistical significance was calculated by the WWC and, where necessary, 
corrects for clustering within classrooms or schools, and for multiple comparisons. For an explanation, see the WWC Tutorial on Mismatch . See the WWC Intervention Rating Scheme for the formulas the WWC used to calculate the 
statistical significance. In the case of the Too Good for Drugs™ report, corrections for clustering and multiple comparisons were needed. 

9. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values 
between -50 and +50, with positive numbers denoting favorable results. 

10. Effect size for this outcome measure was calculated using the odds ratio formula, please see the WWC Technical Working Paper on Effect Size . 

1 1 . The WWC-computed domain average effect size is a simple average rounded to two decimal places. The domain improvement index is calculated from the average effect size. 

12. In both studies reviewed for Too Good for Drugs™ Bacon reported statistically significant positive findings for several student outcomes, but after correcting for clustering and multiple comparisons the WWC found that the differences 
between the groups were not statistically significant. 
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Appendix A4.1 



Summary of end-of-program study findings for the behavior domain 1 









Author’s findings from the study 














Mean outcome 
(standard deviation 2 ) 




WWC calculations 




Outcome measure 3 


Study 

sample 


Sample size 
(students/ 
schools) 4 


Too Good for Comparison 

Drugs™ group group 

(column 1) (column 2) 


Mean difference 5 
(column 1- 
column 2) 


Statistical 
significance 7 
Effect size 6 (at a = 0.05) 


Improvement 

index 8 



Bacon, 2000 (randomized controlled trial) 



Personal and social skills 


Grades 3-4 


6/1,051 


3.71 

(0.78) 


3.50 

(0.66) 


0.21 


0.29 


ns 


+11 


Prosocial behaviors 


Grades 3-4 


6/1,051 


3.79 

(0.87) 


3.37 

(0.72) 


0.42 


0.52 


ns 


+20 


Inappropriate behaviors 


Grades 3-4 


6/1,051 


4.20 

(0.91) 


4.05 

(1.01) 


0.15 


0.16 


ns 


+6 



ns = not statistically significant 



1 . This appendix presents interim and immediate posttest findings for the behavior domain. End-of-program and follow-up scores were used for rating purposes and are presented in Appendix A3.1 . 

2. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. 

3. Bacon (2003) also examined effects on students' perceptions of the harmful effects of drugs and alcohol and students' intentions to smoke tobacco. These outcomes were not included, because they were not relevant to the scope of 
this review. For further information about the scope of this review, please see the haracter Education Protocol . 

4. The WWC received confirmation from the study author that the analysis for pretest equivalence is based on the analysis sample rather than the intent-to-treat sample. This analysis addresses concerns about sample attrition that other- 
wise might affect findings. 

5. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 

6. For an explanation of the effect size calculation, please see the WWC Technical Working Paper on Effect Size . 

7. Statistical significance is the probability that the difference between groups is the result of chance rather than a real difference between the groups. The level of statistical significance was calculated by the WWC and, where neces- 
sary, corrects for clustering within classrooms or schools, and for multiple comparisons. For an explanation, see the WWC Tutorial on Mismatch . See the WWC Intervention Rating Scheme for the formulas the WWC used to calculate the 
statistical significance. In the case of the Too Good for Drugs™ report, corrections for clustering and multiple comparisons were needed. 

8. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values 
between -50 and +50, with positive numbers denoting favorable results. 
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Appendix A4.2 Summary of end-of-program study findings for the knowledge, attitudes, and values domain 1 









Author’s findings from the study 














Mean outcome 
(standard deviation 2 ) 




WWC calculations 




Outcome measure 3 


Study 

sample 


Sample size 
(students/ 
schools) 4 


Too Good for Comparison 

Drugs™ group group 

(column 1) (column 2) 


Mean difference 5 
(column 1- 
column 2) 


Statistical 
significance 7 
Effect size 6 (at a = 0.05) 


Improvement 

index 8 



Intentions for drinking 


Grade 6 


6/1,060 


Bacon, 2000 (randomized controlled trial) 

59 out of 575 58 out of 366 

students students 


1.65 


0.30' 


ns 


+12 


Intention for marijuana 


Grade 6 


6/1,060 


50 out of 589 
students 


45 out of 384 
students 


1.43 


0.22 


ns 


+9 


Resistance skills 


Grade 6 


6/1,060 


4.50 

(0.45) 


4.25 

(0.78) 


0.25 


0.42 


ns 


+16 


Prosocial peers 


Grade 6 


6/1,060 


4.71 

(0.56) 


4.58 

(0.72) 


0.13 


0.21 


ns 


+8 


Locus of control 


Grade 6 


6/1,060 


4.47 

(0.71) 


4.26 

(0.57) 


0.21 


0.32 


ns 


+12 








Bacon, 2003 (randomized controlled trial) 










Emotional competency skills 


Grades 3-4 


6/935 


4.08 


3.94 


0.14 


0.24 


ns 


+10 








(0.57) 


(0.59) 










Social and resistance skills 


Grades 3-4 


6/935 


3.63 


3.49 


0.14 


0.21 


ns 


+8 








(0.68) 


(0.68) 










Goal setting and 


Grades 3-4 


6/935 


4.50 


4.24 


0.26 


0.42 


ns 


+16 


decisionmaking skills 






(0.58) 


(0.66) 











ns = not statistically significant 



1 . This appendix presents interim and immediate posttest findings for measures that fall in the knowledge, attitudes, and values domain. End-of-program and follow-up scores were used for rating purposes and are presented in 
Appendix A3. 2. 

2. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. 

3. Bacon (2003) also examined effects on students’ perceptions of the harmful effects of drugs and alcohol and students’ intentions to smoke tobacco. These outcomes were not included, because they were not relevant to the scope of 
this review. For further information about the scope of this review, please see the haracter Education Protocol . 

4. WWC requested and received from the study author sample sizes for the analysis sample of students for all variables in Bacon (2000) because they were not reported in the study paper. 

5. The WWC received confirmation from the study author that the analysis for pretest equivalence is based on the analysis sample rather than the intent-to-treat sample. This analysis addresses concerns about sample attrition that other- 
wise might affect findings. 

6. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. The numbers in the mean difference column for the intentions for drinking and marijuana use represent 

the odds ratio (ratio between the proportions of the intervention group and comparison group) used to calculate effect size. For an explanation of the effect size calculation, please see the WWC Technical Working Paper on Effect Size . 

(continued) 
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Appendix A4.2 Summary of end-of-program study findings for the knowledge, attitudes, and values domain 1 (continued) 



7. Statistical significance is the probability that the difference between groups is the result of chance rather than a real difference between the groups. The level of statistical significance was calculated by the WWC and where neces- 
sary, corrects for clustering within classrooms or schools, and for multiple comparisons. For an explanation, see the WWC Tutorial on Mismatch . See the WWC Intervention Rating Scheme for the formulas the WWC used to calculate the 
statistical significance. In the case of the Too Good for Drugs™ report, corrections for clustering and multiple comparisons were needed. 

8. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values 
between -50 and +50, with positive numbers denoting favorable results. 
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Appendix A5.1 Rating for the behavior domain 



The WWC rates interventions as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative. 1 

For the outcome domain of behavior, the WWC rated Too Good for Drugs™ as having potentially positive effects. It did not meet the criteria for positive effects, 
because it only had one study. The remaining ratings (mixed effects, no discernible effects, potentially negative effects, and negative effects) were not considered, 
because Too Good for Drugs™ was assigned the highest applicable rating. 

Rating received 

Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1 : At least one study showing a statistically significant or substantively important positive effect, thus qualifying as a positive effect. 

Met. The one study on Too Good for Drugs™ that examined behavior found a substantively important positive effect. 

• Criterion 2: No studies showing a statistically significant negative effect. Fewer or the same number of studies showing indeterminate effects than the number 
showing statistically significant or substantively important positive effects. 

Met. The WWC analysis found no statistically significant negative effects or indeterminate effects in this domain. 

Other ratings considered 

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1 : Two or more studies showing statistically significant positive effects, at least one of which met WWC evidence standards for a strong design. 

Not met. Too Good for Drugs™ had only one evaluation study meeting WWC evidence standards that examined student outcomes in the behavior 
domain. 

• Criterion 2: No studies showing statistically significant or substantively important negative effects. 

Met. No studies sharing statistically significant or substantively important negative effects in this domain. 



1. For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain level effect. The WWC also considers the size of the domain level effect for ratings of 
potentially positive effects. See the WWC Intervention Rating Scheme for a complete description. 
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Appendix A5.2 Rating for the knowledge, attitudes, and values domain 



The WWC rates interventions as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative. 1 

For the outcome domain of knowledge, attitudes, and values, the WWC rated Too Good for Drugs™ as having no discernible effects. It did not meet the criteria for 
other ratings (positive effects, potentially positive effects, mixed effects, potentially negative effects, and negative effects) because none of the studies showed statisti- 
cally significant or substantively important effects. 

Rating received 

No discernible effects: No affirmative evidence of effects. 

• Criterion 1 : None of the studies shows a statistically significant or substantively important effect, either positive or negative. 

Met. Too Good for Drugs™ had two studies meeting WWC evidence standards. Neither study showed a statistically significant or substantively 
important effects. 

Other ratings considered 

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1 : Two or more studies showing statistically significant positive effects, at least one of which met WWC evidence standards for a strong design. 

Not met. Too Good for Drugs™ had two studies meeting WWC evidence standards, both of which met standards for strong design. But neither 
study showed a statistically significant positive effect. 

• Criterion 2: No studies showing statistically significant negative effects. 

Met. The WWC analysis found no statistically significant negative effects in this domain. 

Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1 : At least one study showing a statistically significant or substantively important positive effect, thus qualifying as a positive effect. 

Not met. The WWC analysis found no statistically significant or substantively important positive effects in this domain. 

• Criterion 2: No studies showing a statistically significant negative effect. Fewer or the same number of studies showing indeterminate effects than the number 
showing statistically significant or substantively important positive effects. 

Not met. The WWC analysis found no statistically significant negative effects in this domain. But, while both studies showed indeterminate effects, 
neither showed statistically significant or substantively important positive effects in this domain. 

Mixed effects: Evidence of both positive and negative effects. 

• Criterion 1 : At least one study showing a statistically significant or substantively important positive effect; AND at least one study showing a statistically significant 
negative effect, but no more such studies than the number showing a statistically significant or substantively important positive effect. OR 

Not met. No studies sharing a statistically significant or substantively important positive effect in this domain. 

OR 

• Criterion 2: At least one study showing a statistically significant or substantively important effect AND more studies showing an indeterminate effect than showing 
a statistically significant or substantively important effect. 
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Appendix A5.2 Rating for the knowledge, attitudes, and values domain (continued) 



Not met. No studies sharing a statistically significant or substantively important effect in this domain. 

Potentially negative effects: Evidence of a negative effect with no overriding contrary evidence. 

• Criterion 1 : At least one study showing a statistically significant or substantively important negative effect. 

Not met. No studies sharing a statistically significant or substantively important negative effect in this domain. 

• Criterion 2: No studies showing a statistically significant or substantively important positive effect, OR more studies showing statistically significant or substan- 
tively important negative effects than showing statistically significant or substantively important positive effects. 

Not met. No studies sharing a statistically significant or substantively important positive effect in this domain. 

Negative effects: Strong evidence of a negative effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant negative effects, at least one of which is based on a strong design. 

Not met. The WWC analysis found no statistically significant negative effects in this domain. 

• Criterion 2: No studies showing statistically significant or substantively important positive effects. 

Met. The WWC analysis found no statistically significant or substantively important positive effects in this domain. 



1. For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain level effect. The WWC also considers the size of the domain level effect for ratings of 
potentially positive effects. See the WWC Intervention Rating Scheme for a complete description. 
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